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Abstract — We consider a noisy Slepian-Wolf problem where 
two correlated sources are separately encoded and transmitted 
over two independent binary memoryless symmetric channels. 
Each channel capacity is assumed to be characterized by a single 
parameter which is not known at the transmitter. The receiver 
has knowledge of both the source correlation and the channel 
parameters. We call a system universal if it retains near-capacity 
performance without channel knowledge at the transmitter. 

Kudekar et al. recently showed that terminated low-density 
parity-check (LDPC) convolutional codes (a.k.a. spatially-coupled 
LDPC ensembles) can have belief -propagation thresholds that ap- 
proach their maximum a-posteriori thresholds. This was proven 
for binary erasure channels and shown empirically for binary 
memoryless symmetric channels. They also conjectured that the 
principle of spatial coupling is very general and the phenomenon 
of threshold saturation applies to a very broad class of graphical 
models. In this work, we derive an area theorem for the joint 
decoder and empirically show that threshold saturation occurs 
for this problem. As a result, we demonstrate near-universal 
performance for this problem using the proposed spatially- 
coupled coding system. A similar result is also discussed briefly 
for the 2-user multiple-access channel. 

Index Terms — LDPC codes, spatial coupling, EXIT functions, 
density evolution, correlated sources, non-systematic encoders, 
joint decoding, protograph, area theorem. 

I. Introduction 

The phenomenon of threshold saturation via spatial coupling 
was introduced in [1], [2] to describe the excellent perfor- 
mance of convolutional LDPC codes over binary-input mem- 
oryless symmetric (BMS) channels [3]. Kudekar et al. prove 
that the belief propagation (BP) threshold of the spatially 
coupled ensemble is essentially equal to the maximum a- 
posteriori (MAP) threshold of the underlying ensemble when 
transmission takes place over a binary erasure channel (BEC) 
[1]. Empirical evidence of this phenomenon for BMS channels 
has been observed in [2], [3]. 

The underlying principle behind the impressive performance 
of spatially-coupled codes is very broad and Kudekar et al. 
conjecture that the same phenomenon occurs for more general 
channels. In this work, we consider a noisy Slepian-Wolf 
problem. The outputs of two discrete memoryless correlated 
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sources, (Ut, U2), are transmitted to a central receiver through 
two independent discrete memoryless channels with capacities 
C\ and C2, respectively. In [4], the authors consider the noisy 
Slepian-Wolf problem and observed that the MAP threshold of 
the punctured LDPC(4, 6) ensemble is very close to the capac- 
ity region for transmission over erasure channels. Therefore, 
the phenomenon of threshold saturation motivates the use of 
spatial coupling as a potentially universal coding scheme for 
the noisy Slepian-Wolf problem. In this paper, this observation 
is extended to the 2-user Gaussian multiple access channel 
(MAC). 

We will assume that the channels belong to the same 
channel family, and that each channel can be parameterized by 
a single parameter a (e.g., the erasure probability for erasure 
channels). We also assume that the channel parameters are 
not known at the transmitter. The system model is shown in 
Fig. 1. The two encoders are not allowed to communicate 
and hence they must use independent encoding functions. 
We also assume that both the encoders use identical rates 
R = k/n, i.e., they map k input symbols (Ui and U2) to 
n output symbols (Xi and X2), respectively. The decoder 
receives (Yi,Y2) and computes an estimate of (Ui,U2). 
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Figure 1. System Model 

Reliable transmission over a channel pair (at, a 2 ) is possi- 
ble as long as the pair satisfies the Slepian-Wolf conditions 
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are satisfied. For a given pair of rate-i? encoding functions 
and a joint decoding algorithm, we say that a pair of chan- 
nel parameters (ai,«2) is achievable if the encoder/decoder 



combination can achieve an arbitrarily low probability of error 
as the blocklength n — > oo. As in [5], the achievable channel 
parameter region (ACPR) is defined as the set of all channel 
parameters which are achievable, and the Slepian-Wolf region 
(illustrated in Fig. 2 for erasure channels) is the set of all 
channel parameters (ai,a 2 ) for which (1) is satisfied. Coding 
schemes for which the ACPR is equal to the Slepian-Wolf 
region are said to be universal. Such schemes are important 
because, in some practical situations, it is unreasonable to have 
knowledge of the channel parameters at the transmitter. Hence 
a single coding scheme needs to perform well over a large set 
of channel parameters. 



C2 

l-ff([/ 2 |!7i)-R- 



l-H(U 2 )R- 



sy|mmetiic^:hannel condition 



1-H(U!)R l-H(U 1 \U 2 )R 1 
Figure 2. The Slepian-Wolf region for erasure channels for a rate pair (_R, _R). 

In this paper, we consider the following scenarios: 

1) The channels are binary erasure channels (BECs) and 
the source correlation is modeled through erasures. Let 
Z be a Bernoulli -p random variable. The sources U\ and 
U 2 are defined by 



(Ui,U 2 ) 



i.i.d. Bernoulli h r.v.s, if Z = 



same Bernoulli | r.v. U , if Z = 1 



This gives H{Ui\U 2 ) = H(U 2 \Ui} = 1 - p and 
H(Ui,U 2 ) = 2 — p. In this model, the decoder has 
access to the side information Z. While this model is 
not realistic, it is useful as a toy model that enables us 
to gain a better understanding of the problem. 
2) A more realistic model is one where the channels 
are binary-input additive white Gaussian noise chan- 
nels (BAWGNC) and the source correlation is modeled 
through a virtual correlation channel analogous to a 
binary symmetric channel (BSC). It is useful to visualize 
this correlation by the presence of an auxiliary binary 
symmetric channel (BSC) with parameter 1 — p between 
the sources. In other words, U 2 is the output of a BSC 
with input U\ (a Bernoulli-1/2 random variable) i.e., 
U 2 = Ui + Z. Here Z is a Bernoulli-(l — p) random 
variable and can be thought of as an error. Let H 2 {-) 
denote the binary entropy function. Then, H(U\\U 2 ) = 
H{U 2 \U{) = H 2 (p) and H(U U U 2 ) = 1 + H 2 {p). In 



this model, the side information Z is not available at 
the decoder. 

Although separation between source and channel coding is 
known to be optimal for this problem [6], it can be beneficial to 
take a joint source-channel coding approach (via direct channel 
coding and joint decoding at the receiver) [7]. This problem 
is considered in [8], [9], where the authors choose a code that 
performs well at one point on the Slepian-Wolf region and 
evaluate its performance for different channel parameters. As 
a result, the performance of the code is far from the optimal 
performance for some channel parameters. Even the optimized 
degree profiles of LDPC codes for this problem are far from 
universal [5]. In this paper, we derive the area theorem for the 
joint decoder and compute (G)EXIT curves for transmission 
over symmetric channel conditions. This provides empirical 
evidence that the phenomenon of threshold saturation occurs 
for the noisy Slepian-Wolf problem. Moreover, density evolu- 
tion (DE) results suggest that the spatially-coupled punctured- 
systematic LDPC(4, 6) ensemble is near universal. 

II. Density Evolution and (G)EXIT Curves 

We assume that the sequences Ui and U2 are encoded using 
a punctured-systematic encoder for LDPC codes whose degree 
distribution functions are given by (X,p). The advantages of 
using a punctured-systematic encoder are discussed in [5]. 
Based on standard notation [10], we let \{x) — XiAiX 1-1 
be the degree distribution (from an edge perspective) corre- 
sponding to the variable nodes and p(x) — J^i Pi x% ~ X be the 
degree distribution (from an edge perspective) of the parity- 
check nodes in the decoding graph. The coefficient Aj (resp. 
Pi) gives the fraction of edges that connect to the variable 
nodes (resp. parity-check nodes) of degree i. Likewise, Lj is 
the fraction of variable nodes with degree i. The fraction of 
punctured (i.e., systematic) bits is given by 
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The remainder of this section makes heavy use of the 
terminology and notation from [10] for DE analysis and 
(G)EXIT curves. Let ag and bg denote the L-density 1 of the 
messages emanating from the variable nodes at iteration i, 
corresponding to codes 1 and 2. The density evolution (DE) 
equations [5] can be written as follows 



lS{L (p(M)) + (1 - 7)3bmsc 
7 /(L(p(a,)))+(l- 7 )b; 
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where A(a) = E^a®^" 1 ), L(a) - E^a®^" 1 ), p(a) = 
J2i Pia a( - l ~ 1) ', a BM sc and b BM sc are the densities of the log- 
likelihood ratios received from the channel. The operators © 
and EH are the standard density transformation operators at 
the variable and check nodes respectively [10]. The operator 
/ at the correlation nodes depends on the equivalent channel 

'Assuming that the transmission alphabet is {±1}, the densities are 
conditioned on the transmission of a +1. 



corresponding to the correlation model, as described in [11]. 
For the correlation models considered, one can derive a 
generalized symmetry condition that allows the function / to 
be chosen so that DE can be performed under the all-zero 
codeword assumption. 

In Sections II-A and II-B, we describe the (G)EXIT curves 
for the joint decoder. For simplicity, we consider (G)EXIT 
curves for symmetric channel conditions throughout this work. 
In this case, the DE equations collapse into the recursion 
a? + i = D(aBMSC,af). Similar to single user channels, the 
area theorem for the joint decoder can be used to compute 
an upper bound on the MAP threshold of the joint decoder. 
For example, this technique was applied to the joint decoding 
of a finite-state channel and an LDPC code in [12]. It has 
been observed that this upper bound is tight for regular LDPC 
ensembles transmitted over the BEC [13]. For asymmetric 
channel conditions, we can define 2-dimensional (G)EXIT 
surfaces analogously and the area theorem gives outer bounds 
on the MAP boundary. As described in [10], it is useful to 
assume that each bit of user 1 (user 2) has been transmitted 
through a channel with parameter or^ (a^X an d suppose 
that these parameters are differentiable functions of a common 
parameter a. The area theorem follows trivially from the 
definition of the MAP (G)EXIT function and is given by 
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A. Erasure Correlation 



For erasure correlation with probability p, there is a parity- 
check at the correlation-node with probability p and with prob- 
ability 1—p there is no parity-check, so /(a) = (1 — p) +pa. 
For simplicity, we consider the EXIT curves for the case when 
the channel erasure probabilities for both users are equal. 
The extended belief propagation (EBP) EXIT curve for the 
joint decoder with symmetric channel conditions, is given in 
parametric form by 



(e(x),L(l - p(l - x))), x S [0, 1], where 
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The EBP EXIT curve and the MAP threshold for the punctured 
LDPC(4, 6) ensemble are shown in Fig. 4. The MAP threshold 
at symmetric channel conditions is g MAP « 0.6245, while the 
Slepian-Wolf bound is e = 0.625. 

B. BSC Correlation 

Using the generalized symmetry condition for this model 
results in the correlation node function /(a) ~ a B sc( P ) ffl a. It 
turns out that the GEXIT kernel for the joint decoder is the 
same as that of the standard BAWGNC, given by 



£ a BAWGNC(h) ^y*j - 



1 + e z +y 
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where a is the unique positive number such that the BAWGNC 
with noise variance a 2 has entropy h. Let 



G(a BA wGNC(h),a) = J a(y)^ A ™> (y)dy 

be the GEXIT functional applied to the density a. For 
each fixed point (not necessarily stable) of density evolution 
(a B AWGNC(h),a), satisfying a = D(a BA wGNC(h), a), the point 
(h, G(a B AwGNC(h): a)) lies on the EBP GEXIT curve. The EBP 
GEXIT curve is the set of all these points and can be computed 
numerically as outlined in [14]. The EBP GEXIT curve and 
the MAP threshold for the punctured LDPC (4, 6) ensemble are 
shown in Fig. 5. The MAP threshold for symmetric channel 
conditions is h MAP pa 0.6324 and the Slepian-Wolf bound is 
h « 0.6328. 

III. Spatial Coupling 

Spatial coupling is best described by the (l,r,L) ensemble 
through a protograph [1], [15]. We briefly review the proto- 
graph structure at the joint decoder here. Consider the proto- 
graph of a standard LDPC (4, 6) ensemble. There are two check 
nodes and three variable nodes. For each user, take a collection 
of (2L + 1) protographs at positions [— L, L] = {— L, ■ ■ ■ ,L} 
and couple them as described in [1]. One variable node at 
each position i 6 [— L, L] from the first user is punctured and 
connected to a punctured variable node at the same position of 
the second user. The resulting protograph, shown in Fig. 3, is 
then expanded M times to form the parity-check matrix of the 
joint system. This structure is fundamental to the phenomenon 
of threshold saturation observed at the joint decoder. It is sim- 
ply not sufficient to use spatially-coupled codes with random 
connections between the information nodes. Such a coupling 
will only result in pushing the threshold of the component 
codes to the MAP threshold, but may have little effect on the 
BP threshold of the joint system. 





2L + 1 

Figure 3. Protograph of the joint decoder 

Although this ensemble is very instructive in understanding 
the universality of spatially-coupled codes, the EBP curves 
for this ensemble exhibit wiggles around the MAP threshold 
(similar to the single user channels as discussed in [1]) , for 
the case of transmission over erasure channels. The magnitude 
of these wiggles appears to remain constant with increasing L 
and their presence implies that the BP threshold is smaller than 



the MAP threshold of the underlying ensemble. Therefore, 
the (4, 6, L) ensemble cannot be universal. To overcome this, 
we use the (l,r,L,w) ensemble introduced in [1] for the 
remainder of this work. 

A. The (l,r,L,w) ensemble 

The (I, r, L, w) spatially-coupled ensemble can be described 
as follows: Place M variable nodes at each position in \—L, L], 
The check nodes are placed at positions [— L, L + w — 1], with 
\M check nodes at each position. The connections are made 
as described in [1]. This procedure generates a Tanner graph 
for the (l,r,L,w) ensemble. 

For this work we consider codes of rate 1 /3, punctured to a 
rate 1/2. Two such graphs (generated by the above procedure) 
are taken and 2M/3 variable nodes (M/3 from each graph) 
at each position are connected by a random (uniform) per- 
mutation of size M/3 via correlation nodes. This procedure 
ensures that all the variable node positions are symmetric (as 
opposed to Fig. 3) with respect to puncturing and correlation, 
enabling us to write down the density evolution (DE) equations 
as described in the following section. 

B. Density Evolution of the (l,r,L,w) Ensemble 

Let and b\ denote the average density emitted by the 
variable node at position i, at iteration I, for codes 1 and 2 
respectively. Let A +oc denote the delta function at +oo and set 
ap — = A+oo for i </ [-L, L\. The channel densities for 
codes 1 and 2 are denoted by sbmsc and b B MSC respectively. 
All the above densities are L-densities conditioned on the 
transmission of the all-zero codeword (see Section II). We 
consider the parallel schedule for each user (as described in 
[1]) and update the correlation nodes before proceeding to the 
next iteration. Let us define 

_ H(r-1)\ «(!-!) 

g(xi- w +i, ■ ■ ■ , x i+ro _i) = — \ — > y-i+j-k 

\ in — \ in — 



— JIT -punc (4,6) 

— MAP - punc (4, 6) 

— JIT - L = 16, w = 2 

— JIT-L = 32,11 = 4 
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Figure 4. EBP EXIT curves of the (4, 6, L, w) and (4, 6) ensembles for 
transmission over erasure channels with erasure correlated sources. 
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Figure 5. EBP GEXIT curves of the (4, 6,L,u>) and (4,6) ensembles 
for transmission over AWGN channels which BSC correlation between the 



We can use the procedure outlined in [14] to compute both 
the stable and unstable fixed points which satisfy (4). Define 
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The DE equations for the joint spatially-coupled system can 
be written as 

af +1) =[7/(r(bH +1) -,bit- 1 ))+(l-7) £ 
a(a {e) ••• a W ) 

a(b {t) ••• b W ) 

for i G [— L, L\. For a further discussion of the DE equations 
for the (/, r, L, w) spatially-coupled ensembles on BMS chan- 
nels, see [2]. Let a = (a_£, • • • ,a£). The fixed points of DE 
for symmetric channel conditions are given by (a BM sc(h);<i), 
which satisfy 

ai = [7/ {T{a l - W+1 , ■■■ , a i+w -i)) + (1 - 7) a BMSC(h)] © 

gi^i-w+l, ■ ■ • , 3i+w-l)- (4) 
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The EBP GEXIT curve is the set of points 
(h, G(a BM sc(h),3.))- The resulting curves for the erasure 
channel with erasure correlated sources are shown in Fig. 4 
and those for the AWGN channel with BSC correlated 
sources are shown in Fig. 5. These curves are very similar 
to the single user case and demonstrate the phenomenon 
of threshold saturation at the joint decoder, for symmetric 
channel conditions. For channel parameters not on the 
symmetric line, these plots imply threshold saturation towards 
the MAP boundary. 

IV. The 2-user Gaussian MAC 

Consider a 2-user additive Gaussian multiple access channel 
(MAC), given by 

Y = hiXi + h 2 X 2 + Z, 




hi ei SNRi 

Figure 6. DE ACPR of a spatially coupled Figure 7. DE ACPR of the spatially coupled punc- Figure 8. DE ACPR of the spatially coupled 
(3, 6, 32, 4) LDPC code for the two user Gaussian tured (4,6,64,10) LDPC and the regular punc- punctured (4,6,64,10) LDPC and the regular 
MAC. The DE results for the regular LDPC(3, 6) tured LDPC(4, 6) ensembles for transmission over punctured LDPC(4, 6) ensembles for transmission 
ensemble are shown for comparison. erasure channels with erasure correlated sources, over AWGN channels with BSC correlated sources. 



with Z ~ Af(0,l), X X ,X 2 e {±1} and hi, ha G R. We 
assume that the fading coefficients are not known at the trans- 
mitter and we consider the notion of universality with respect 
to fading coefficients. The factor graph of the joint decoder 
consists of two single user Tanner graphs, whose variable 
nodes are connected through a function node [10, p. 308]. 
Using the notation described in Section II, the DE equation for 
the joint decoder with symmetric fading coefficients is given 
by 

a^+i = f{L (fi(s e )) , a BA wGNc) © A(p(a £ )). 

Here, / denotes the operation at the function node for trans- 
formation of densities, and is chosen under the assumption of 
transmission of a random coset of the LDPC code. Preliminary 
DE results show that spatial coupling allows for near universal 
performance on the Gaussian MAC (see Fig. 6). In our future 
work, we will derive the area theorem (with an appropriate 
GEXIT kernel) for this problem to formalize this result. 

V. Results and Concluding Remarks 

It was shown in [1], that for transmission over erasure 
channels, the BP threshold of spatially-coupled ensembles 
is essentially equal to the MAP threshold of the underlying 
ensemble. This was observed numerically for general BMS 
channels in [2], [3]. In this work, we numerically show that 
the phenomenon of threshold saturation is very general and can 
provide universality for multi-user scenarios. In particular, we 
considered the noisy Slepian-Wolf problem and showed that 
spatial coupling boosts the BP threshold of the joint decoder 
to the MAP threshold of the underlying ensemble. The density 
evolution ACPRs for the two scenarios considered in this paper 
are shown in Fig. 7 and 8. These figures show that spatially 
coupled ensembles are near universal for this problem. The 
analytic proof of this result remains an open problem. Such 
a proof would essentially show that it is possible to achieve 
universality for the noisy Slepian-Wolf problem under iterative 
decoding. 
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